Consistent Selection of Tuning Parameters via Variable Selection Stability ( Supplementary Material )

In this supplementary material, we provide Lemmas 2 and 3 and their proofs. Suppose that x 1 ,. .. , x n are i.i.d. from a probability distribution with mean 0 and finite covariance matrix C = (C jk). Assumption S1: Assume that x 1 has finite fourth moment, that is, E(x 1i x 1 j x 1k x 1l) is finite for any 1 ≤ i, j, k, l ≤ p. Lemma 2 Suppose that Assumption S1 is met. Assumptions 1 and 2 are satisfied by the lasso regression and the SCAD with r n = n −1/2 and s n = o(1) under the assumptions in Zhao and Yu (2006) or Fan and Li (2001), and by the adaptive lasso with r n = n −1 and s n = n −1/2 under the assumptions in Zou (2006), on the random splitting subsamples generated in Algorithm 1.